• All Files

 

Change Data Capture using Spark SQL

 

Open solution

 

Category	Talend specific, Spark
Prerequisites	Talend Data Integration Basics, Talend Big Data Basics, Talend Big Data - Spark Batch, Context Management Common Joblets
Third-party software	Spark SQL, Hadoop cluster
Description	The Spark SQL CDC template Job uses Spark SQL to compare two datasets and identify records that need to be inserted, updated, and deleted after comparing with the last Job execution.